CloudWatch LogsからKinesis Data Fireshoseサブスクリプションフィルターを使ってS3へログを転送した時のログ形式を、オリジナルのログ形式に戻す

CloudWatch LogsからKinesis Data Fireshoseサブスクリプションフィルターを使ってS3へログを転送した時のログ形式を、オリジナルのログ形式に戻す

#AWS Lambda
#CloudWatch Logs
#Amazon Kinesis
#Amazon Data Firehose
#AWS
いわさ

いわさ

facebook logohatena logotwitter logo
Clock Icon2021.07.25

この記事は公開されてから1年以上経過しています。情報が古い可能性がありますので、ご注意ください。

いわさです。

仮想マシンのログファイルをCloudWatchエージェント等で、CloudWatch Logsへ転送するケース多いと思います。
その後、Kinesis Data Firehoseサブスクリプションフィルターを使うことでS3で長期的にログファイルの保存・アーカイブを行うことが出来ます。

しかし、CloudWatch Logs と Kinesis Data Firehoseの連携だけだとオリジナルのログデータにCloudWatch Logsに関する情報が追加されています。

この問題を解決し元のログデータに戻してS3へ格納するためには、Lambda関数が必要となります。
Lambda関数が必要らしいという情報まで辿り着いたところで、「Lambdaが必要...うっ頭が」という方もいらっしゃるかもしれません。

しかし、Lambdaにはブループリントが用意されており、前述のログ変換についても用意されています。
そしてコーディング不要でそのまま使い始めることが可能です。

変換なしのログ形式

Amazon Linux 2 の/var/messagesログと、Windows Server 2019 の システムログ で試しています。

{
    "messageType": "DATA_MESSAGE",
    "owner": "123456789012",
    "logGroup": "/var/log/messages",
    "logStream": "i-0ea9977f466536e50",
    "subscriptionFilters": [
        "iwasa-cloudwatchlogs-MessagesLogSubscriptionFilter-1LS8C9RAT1KPH"
    ],
    "logEvents": [
        {
            "id": "36278389327885475969123320312855138996605835115192188928",
            "timestamp": 1626779240107,
            "message": "Jul 20 11:07:15 ip-172-31-33-72 dhclient[2753]: XMT: Solicit on eth0, interval 108820ms."
        }
    ]
}{
    "messageType": "DATA_MESSAGE",
    "owner": "123456789012",
    "logGroup": "/var/log/messages",
    "logStream": "i-0ea9977f466536e50",
    "subscriptionFilters": [
        "iwasa-cloudwatchlogs-MessagesLogSubscriptionFilter-1LS8C9RAT1KPH"
    ],
    "logEvents": [
        {
            "id": "36278391754652568473225730706285194146673381764925489152",
            "timestamp": 1626779348927,
            "message": "Jul 20 11:09:04 ip-172-31-33-72 dhclient[2753]: XMT: Solicit on eth0, interval 129370ms."
        },
        {
            "id": "36278391754652568473225730706285194146673381764925489153",
            "timestamp": 1626779348927,
            "message": "Jul 20 11:09:08 ip-172-31-33-72 dhclient[2706]: DHCPREQUEST on eth0 to 172.31.32.1 port 67 (xid=0x2833da31)"
        },
        {
            "id": "36278391754652568473225730706285194146673381764925489154",
            "timestamp": 1626779348927,
            "message": "Jul 20 11:09:08 ip-172-31-33-72 dhclient[2706]: DHCPACK from 172.31.32.1 (xid=0x2833da31)"
        },
        {
            "id": "36278391754652568473225730706285194146673381764925489155",
            "timestamp": 1626779348927,
            "message": "Jul 20 11:09:08 ip-172-31-33-72 dhclient[2706]: bound to 172.31.33.72 -- renewal in 1763 seconds."
        },
        {
            "id": "36278391754652568473225730706285194146673381764925489156",
            "timestamp": 1626779348927,
            "message": "Jul 20 11:09:08 ip-172-31-33-72 ec2net: [get_meta] Querying IMDS for meta-data/network/interfaces/macs/06:df:a1:27:97:ff/local-ipv4s"
        },
        {
            "id": "36278391754652568473225730706285194146673381764925489157",
            "timestamp": 1626779348927,
            "message": "Jul 20 11:09:08 ip-172-31-33-72 ec2net: [get_meta] Getting token for IMDSv2."
        },
        {
            "id": "36278391754652568473225730706285194146673381764925489158",
            "timestamp": 1626779348927,
            "message": "Jul 20 11:09:08 ip-172-31-33-72 ec2net: [get_meta] Trying to get http://169.254.169.254/latest/meta-data/network/interfaces/macs/06:df:a1:27:97:ff/local-ipv4s"
        },
        {
            "id": "36278391847869683403083735437904496526343532859923628039",
            "timestamp": 1626779353107,
            "message": "Jul 20 11:09:08 ip-172-31-33-72 ec2net: [remove_aliases] Removing aliases of eth0"
        }
    ]
}

{
    "messageType": "DATA_MESSAGE",
    "owner": "123456789012",
    "logGroup": "Windows/System",
    "logStream": "i-095dd791cbb79514d",
    "subscriptionFilters": [
        "iwasa-cloudwatchlogs-WinSystemLogSubscriptionFilter-UEIJGPYHC0SE"
    ],
    "logEvents": [
        {
            "id": "36277846857374658513237166582882787328672537367783538688",
            "timestamp": 1626754914888,
            "message": "<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Service Control Manager' Guid='{555908d1-a6d7-4695-8e1e-26931d2012f4}' EventSourceName='Service Control Manager'/><EventID Qualifiers='16384'>7036</EventID><Version>0</Version><Level>4</Level><Task>0</Task><Opcode>0</Opcode><Keywords>0x8080000000000000</Keywords><TimeCreated SystemTime='2021-07-20T04:21:54.888493400Z'/><EventRecordID>68186</EventRecordID><Correlation/><Execution ProcessID='764' ThreadID='3524'/><Channel>System</Channel><Computer>EC2AMAZ-P1T7GOG</Computer><Security/></System><EventData><Data Name='param1'>Windows Installer</Data><Data Name='param2'>stopped</Data><Binary>6D00730069007300650072007600650072002F0031000000</Binary></EventData><RenderingInfo Culture='en-US'><Message>The Windows Installer service entered the stopped state.</Message><Level>Information</Level><Task></Task><Opcode></Opcode><Channel></Channel><Provider>Microsoft-Windows-Service Control Manager</Provider><Keywords><Keyword>Classic</Keyword></Keywords></RenderingInfo></Event>"
        }
    ]
}{
    "messageType": "DATA_MESSAGE",
    "owner": "123456789012",
    "logGroup": "Windows/System",
    "logStream": "i-095dd791cbb79514d",
    "subscriptionFilters": [
        "iwasa-cloudwatchlogs-WinSystemLogSubscriptionFilter-UEIJGPYHC0SE"
    ],
    "logEvents": [
        {
            "id": "36277847552221277409054322464316465099275899847217315840",
            "timestamp": 1626754946046,
            "message": "<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Service Control Manager' Guid='{555908d1-a6d7-4695-8e1e-26931d2012f4}' EventSourceName='Service Control Manager'/><EventID Qualifiers='16384'>7036</EventID><Version>0</Version><Level>4</Level><Task>0</Task><Opcode>0</Opcode><Keywords>0x8080000000000000</Keywords><TimeCreated SystemTime='2021-07-20T04:22:26.046148800Z'/><EventRecordID>68187</EventRecordID><Correlation/><Execution ProcessID='764' ThreadID='3868'/><Channel>System</Channel><Computer>EC2AMAZ-P1T7GOG</Computer><Security/></System><EventData><Data Name='param1'>AppX Deployment Service (AppXSVC)</Data><Data Name='param2'>stopped</Data><Binary>41007000700058005300760063002F0031000000</Binary></EventData><RenderingInfo Culture='en-US'><Message>The AppX Deployment Service (AppXSVC) service entered the stopped state.</Message><Level>Information</Level><Task></Task><Opcode></Opcode><Channel></Channel><Provider>Microsoft-Windows-Service Control Manager</Provider><Keywords><Keyword>Classic</Keyword></Keywords></RenderingInfo></Event>"
        }
    ]
}

Json形式で加工されます。
そして、logEventsに生データが格納されています。

変換用のLambda関数 "Kinesis Data Firehose Cloudwatch Logs Processor" を使う

では冒頭で述べた、ブループリントを使ったLambda関数の作成を行ってみましょう。

Kinesis Data FirehoseでLambda関数変換を設定するデリバリーストリームのDetails->Editを押下します。
そして、Transform source records with AWS LambdaをEnabledに設定し、Create newボタンで関数を作成します。

ブループリント選択画面が表示されます。
ここでKinesis Data Firehose Cloudwatch Logs Processorを選択します。

選択すると、Lambda関数の作成画面に遷移します。

Lambda関数にロールを設定し、コードは変更しなくて良いです。
ただし、関数の設定からタイムアウトの時間だけ変更しましょう。デフォルトは3秒になっているのでKinesis Data Firehoseの画面で推奨されるように、1分以上に変更します。

なお、このLambda関数は Kinesis Data Firehose側から設定しなくとも、Lambdaからの関数新規作成時でも選択が可能です。

Kinesisに戻って関数を設定したら完了です。

変換によって扱いやすいログ形式に。Athenaでもクエリしやすい

では、変換されS3へ転送されたログを確認してみましょう。

Jul 20 11:38:49 ip-172-31-33-72 dhclient[2752]: XMT: Solicit on eth0, interval 113540ms.
Jul 20 11:40:01 ip-172-31-33-72 systemd: Created slice User Slice of root.
Jul 20 11:40:01 ip-172-31-33-72 systemd: Started Session 2 of user root.
Jul 20 11:40:01 ip-172-31-33-72 systemd: Removed slice User Slice of root.
Jul 20 11:40:43 ip-172-31-33-72 dhclient[2752]: XMT: Solicit on eth0, interval 120010ms.
Jul 20 11:42:12 ip-172-31-33-72 systemd: Starting Cleanup of Temporary Directories...
Jul 20 11:42:12 ip-172-31-33-72 systemd: Started Cleanup of Temporary Directories.
Jul 20 11:42:43 ip-172-31-33-72 dhclient[2752]: XMT: Solicit on eth0, interval 113670ms.

<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Service Control Manager' Guid='{555908d1-a6d7-4695-8e1e-26931d2012f4}' EventSourceName='Service Control Manager'/><EventID Qualifiers='16384'>7036</EventID><Version>0</Version><Level>4</Level><Task>0</Task><Opcode>0</Opcode><Keywords>0x8080000000000000</Keywords><TimeCreated SystemTime='2021-07-20T11:38:27.903276900Z'/><EventRecordID>68821</EventRecordID><Correlation/><Execution ProcessID='736' ThreadID='752'/><Channel>System</Channel><Computer>EC2AMAZ-P1T7GOG</Computer><Security/></System><EventData><Data Name='param1'>Remote Registry</Data><Data Name='param2'>stopped</Data><Binary>520065006D006F0074006500520065006700690073007400720079002F0031000000</Binary></EventData><RenderingInfo Culture='en-US'><Message>The Remote Registry service entered the stopped state.</Message><Level>Information</Level><Task></Task><Opcode></Opcode><Channel></Channel><Provider>Microsoft-Windows-Service Control Manager</Provider><Keywords><Keyword>Classic</Keyword></Keywords></RenderingInfo></Event>
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Service Control Manager' Guid='{555908d1-a6d7-4695-8e1e-26931d2012f4}' EventSourceName='Service Control Manager'/><EventID Qualifiers='16384'>7036</EventID><Version>0</Version><Level>4</Level><Task>0</Task><Opcode>0</Opcode><Keywords>0x8080000000000000</Keywords><TimeCreated SystemTime='2021-07-20T11:38:31.696446500Z'/><EventRecordID>68822</EventRecordID><Correlation/><Execution ProcessID='736' ThreadID='1704'/><Channel>System</Channel><Computer>EC2AMAZ-P1T7GOG</Computer><Security/></System><EventData><Data Name='param1'>Windows Update</Data><Data Name='param2'>running</Data><Binary>770075006100750073006500720076002F0034000000</Binary></EventData><RenderingInfo Culture='en-US'><Message>The Windows Update service entered the running state.</Message><Level>Information</Level><Task></Task><Opcode></Opcode><Channel></Channel><Provider>Microsoft-Windows-Service Control Manager</Provider><Keywords><Keyword>Classic</Keyword></Keywords></RenderingInfo></Event>
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Service Control Manager' Guid='{555908d1-a6d7-4695-8e1e-26931d2012f4}' EventSourceName='Service Control Manager'/><EventID Qualifiers='16384'>7036</EventID><Version>0</Version><Level>4</Level><Task>0</Task><Opcode>0</Opcode><Keywords>0x8080000000000000</Keywords><TimeCreated SystemTime='2021-07-20T11:38:36.244351400Z'/><EventRecordID>68823</EventRecordID><Correlation/><Execution ProcessID='736' ThreadID='752'/><Channel>System</Channel><Computer>EC2AMAZ-P1T7GOG</Computer><Security/></System><EventData><Data Name='param1'>Delivery Optimization</Data><Data Name='param2'>running</Data><Binary>44006F005300760063002F0034000000</Binary></EventData><RenderingInfo Culture='en-US'><Message>The Delivery Optimization service entered the running state.</Message><Level>Information</Level><Task></Task><Opcode></Opcode><Channel></Channel><Provider>Microsoft-Windows-Service Control Manager</Provider><Keywords><Keyword>Classic</Keyword></Keywords></RenderingInfo></Event>
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Service Control Manager' Guid='{555908d1-a6d7-4695-8e1e-26931d2012f4}' EventSourceName='Service Control Manager'/><EventID Qualifiers='16384'>7036</EventID><Version>0</Version><Level>4</Level><Task>0</Task><Opcode>0</Opcode><Keywords>0x8080000000000000</Keywords><TimeCreated SystemTime='2021-07-20T11:38:36.356442400Z'/><EventRecordID>68824</EventRecordID><Correlation/><Execution ProcessID='736' ThreadID='1916'/><Channel>System</Channel><Computer>EC2AMAZ-P1T7GOG</Computer><Security/></System><EventData><Data Name='param1'>Storage Service</Data><Data Name='param2'>running</Data><Binary>530074006F0072005300760063002F0034000000</Binary></EventData><RenderingInfo Culture='en-US'><Message>The Storage Service service entered the running state.</Message><Level>Information</Level><Task></Task><Opcode></Opcode><Channel></Channel><Provider>Microsoft-Windows-Service Control Manager</Provider><Keywords><Keyword>Classic</Keyword></Keywords></RenderingInfo></Event>
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Service Control Manager' Guid='{555908d1-a6d7-4695-8e1e-26931d2012f4}' EventSourceName='Service Control Manager'/><EventID Qualifiers='16384'>7036</EventID><Version>0</Version><Level>4</Level><Task>0</Task><Opcode>0</Opcode><Keywords>0x8080000000000000</Keywords><TimeCreated SystemTime='2021-07-20T11:38:36.595342700Z'/><EventRecordID>68825</EventRecordID><Correlation/><Execution ProcessID='736' ThreadID='752'/><Channel>System</Channel><Computer>EC2AMAZ-P1T7GOG</Computer><Security/></System><EventData><Data Name='param1'>State Repository Service</Data><Data Name='param2'>running</Data><Binary>530074006100740065005200650070006F007300690074006F00720079002F0034000000</Binary></EventData><RenderingInfo Culture='en-US'><Message>The State Repository Service service entered the running state.</Message><Level>Information</Level><Task></Task><Opcode></Opcode><Channel></Channel><Provider>Microsoft-Windows-Service Control Manager</Provider><Keywords><Keyword>Classic</Keyword></Keywords></RenderingInfo></Event>
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Service Control Manager' Guid='{555908d1-a6d7-4695-8e1e-26931d2012f4}' EventSourceName='Service Control Manager'/><EventID Qualifiers='16384'>7036</EventID><Version>0</Version><Level>4</Level><Task>0</Task><Opcode>0</Opcode><Keywords>0x8080000000000000</Keywords><TimeCreated SystemTime='2021-07-20T11:38:36.676303900Z'/><EventRecordID>68826</EventRecordID><Correlation/><Execution ProcessID='736' ThreadID='752'/><Channel>System</Channel><Computer>EC2AMAZ-P1T7GOG</Computer><Security/></System><EventData><Data Name='param1'>AppX Deployment Service (AppXSVC)</Data><Data Name='param2'>running</Data><Binary>41007000700058005300760063002F0034000000</Binary></EventData><RenderingInfo Culture='en-US'><Message>The AppX Deployment Service (AppXSVC) service entered the running state.</Message><Level>Information</Level><Task></Task><Opcode></Opcode><Channel></Channel><Provider>Microsoft-Windows-Service Control Manager</Provider><Keywords><Keyword>Classic</Keyword></Keywords></RenderingInfo></Event>
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Service Control Manager' Guid='{555908d1-a6d7-4695-8e1e-26931d2012f4}' EventSourceName='Service Control Manager'/><EventID Qualifiers='16384'>7036</EventID><Version>0</Version><Level>4</Level><Task>0</Task><Opcode>0</Opcode><Keywords>0x8080000000000000</Keywords><TimeCreated SystemTime='2021-07-20T11:38:36.709681600Z'/><EventRecordID>68827</EventRecordID><Correlation/><Execution ProcessID='736' ThreadID='1916'/><Channel>System</Channel><Computer>EC2AMAZ-P1T7GOG</Computer><Security/></System><EventData><Data Name='param1'>Client License Service (ClipSVC)</Data><Data Name='param2'>running</Data><Binary>43006C00690070005300560043002F0034000000</Binary></EventData><RenderingInfo Culture='en-US'><Message>The Client License Service (ClipSVC) service entered the running state.</Message><Level>Information</Level><Task></Task><Opcode></Opcode><Channel></Channel><Provider>Microsoft-Windows-Service Control Manager</Provider><Keywords><Keyword>Classic</Keyword></Keywords></RenderingInfo></Event>
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-WindowsUpdateClient' Guid='{945a8954-c147-4acd-923f-40c45405a658}'/><EventID>44</EventID><Version>1</Version><Level>4</Level><Task>1</Task><Opcode>12</Opcode><Keywords>0x8000000000002004</Keywords><TimeCreated SystemTime='2021-07-20T11:38:39.888076300Z'/><EventRecordID>68828</EventRecordID><Correlation ActivityID='{5763a5f8-7d5a-0000-46ab-63575a7dd701}'/><Execution ProcessID='1244' ThreadID='2360'/><Channel>System</Channel><Computer>EC2AMAZ-P1T7GOG</Computer><Security UserID='S-1-5-18'/></System><EventData><Data Name='updateTitle'>Security Intelligence Update for Microsoft Defender Antivirus - KB2267602 (Version 1.343.1338.0)</Data><Data Name='updateGuid'>{7be15a28-ba35-417c-a4eb-043abc05d9dd}</Data><Data Name='updateRevisionNumber'>200</Data></EventData><RenderingInfo Culture='en-US'><Message>Windows Update started downloading an update.</Message><Level>Information</Level><Task>Windows Update Agent</Task><Opcode>Download</Opcode><Channel>System</Channel><Provider>Microsoft-Windows-WindowsUpdateClient</Provider><Keywords><Keyword>Download</Keyword><Keyword>Started</Keyword></Keywords></RenderingInfo></Event>
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-WindowsUpdateClient' Guid='{945a8954-c147-4acd-923f-40c45405a658}'/><EventID>43</EventID><Version>1</Version><Level>4</Level><Task>1</Task><Opcode>13</Opcode><Keywords>0x8000000000002008</Keywords><TimeCreated SystemTime='2021-07-20T11:38:39.888079300Z'/><EventRecordID>68829</EventRecordID><Correlation ActivityID='{5763a5f8-7d5a-0000-46ab-63575a7dd701}'/><Execution ProcessID='1244' ThreadID='2360'/><Channel>System</Channel><Computer>EC2AMAZ-P1T7GOG</Computer><Security UserID='S-1-5-18'/></System><EventData><Data Name='updateTitle'>Security Intelligence Update for Microsoft Defender Antivirus - KB2267602 (Version 1.343.1338.0)</Data><Data Name='updateGuid'>{7be15a28-ba35-417c-a4eb-043abc05d9dd}</Data><Data Name='updateRevisionNumber'>200</Data></EventData><RenderingInfo Culture='en-US'><Message>Installation Started: Windows has started installing the following update: Security Intelligence Update for Microsoft Defender Antivirus - KB2267602 (Version 1.343.1338.0)</Message><Level>Information</Level><Task>Windows Update Agent</Task><Opcode>Installation</Opcode><Channel>System</Channel><Provider>Microsoft-Windows-WindowsUpdateClient</Provider><Keywords><Keyword>Installation</Keyword><Keyword>Started</Keyword></Keywords></RenderingInfo></Event>
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-WindowsUpdateClient' Guid='{945a8954-c147-4acd-923f-40c45405a658}'/><EventID>19</EventID><Version>1</Version><Level>4</Level><Task>1</Task><Opcode>13</Opcode><Keywords>0x8000000000000018</Keywords><TimeCreated SystemTime='2021-07-20T11:38:58.584445900Z'/><EventRecordID>68830</EventRecordID><Correlation ActivityID='{5763a5f8-7d5a-0000-46ab-63575a7dd701}'/><Execution ProcessID='1244' ThreadID='2360'/><Channel>System</Channel><Computer>EC2AMAZ-P1T7GOG</Computer><Security UserID='S-1-5-18'/></System><EventData><Data Name='updateTitle'>Security Intelligence Update for Microsoft Defender Antivirus - KB2267602 (Version 1.343.1338.0)</Data><Data Name='updateGuid'>{7be15a28-ba35-417c-a4eb-043abc05d9dd}</Data><Data Name='updateRevisionNumber'>200</Data><Data Name='serviceGuid'>{9482f4b4-e343-43b6-b170-9a65bc822c77}</Data></EventData><RenderingInfo Culture='en-US'><Message>Installation Successful: Windows successfully installed the following update: Security Intelligence Update for Microsoft Defender Antivirus - KB2267602 (Version 1.343.1338.0)</Message><Level>Information</Level><Task>Windows Update Agent</Task><Opcode>Installation</Opcode><Channel>System</Channel><Provider>Microsoft-Windows-WindowsUpdateClient</Provider><Keywords><Keyword>Installation</Keyword><Keyword>Success</Keyword></Keywords></RenderingInfo></Event>
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Time-Service' Guid='{06edcfeb-0fd0-4e53-acca-a6f8bbf81bcb}'/><EventID>37</EventID><Version>0</Version><Level>4</Level><Task>0</Task><Opcode>0</Opcode><Keywords>0x8000000000000000</Keywords><TimeCreated SystemTime='2021-07-20T11:40:00.027722700Z'/><EventRecordID>68831</EventRecordID><Correlation/><Execution ProcessID='1368' ThreadID='1736'/><Channel>System</Channel><Computer>EC2AMAZ-P1T7GOG</Computer><Security UserID='S-1-5-19'/></System><EventData Name='TMP_EVENT_TIME_SOURCE_REACHABLE'><Data Name='TimeSource'>169.254.169.123,0x9 (ntp.m|0x9|0.0.0.0:123-&gt;169.254.169.123:123)</Data></EventData><RenderingInfo Culture='en-US'><Message>The time provider NtpClient is currently receiving valid time data from 169.254.169.123,0x9 (ntp.m|0x9|0.0.0.0:123-&gt;169.254.169.123:123).</Message><Level>Information</Level><Task></Task><Opcode>Info</Opcode><Channel>System</Channel><Provider>Microsoft-Windows-Time-Service</Provider><Keywords></Keywords></RenderingInfo></Event>
<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Time-Service' Guid='{06edcfeb-0fd0-4e53-acca-a6f8bbf81bcb}'/><EventID>35</EventID><Version>0</Version><Level>4</Level><Task>0</Task><Opcode>0</Opcode><Keywords>0x8000000000000000</Keywords><TimeCreated SystemTime='2021-07-20T11:40:00.043083600Z'/><EventRecordID>68832</EventRecordID><Correlation/><Execution ProcessID='1368' ThreadID='1736'/><Channel>System</Channel><Computer>EC2AMAZ-P1T7GOG</Computer><Security UserID='S-1-5-19'/></System><EventData Name='TMP_EVENT_TIME_SOURCE_CHOSEN'><Data Name='TimeSource'>169.254.169.123,0x9 (ntp.m|0x9|0.0.0.0:123-&gt;169.254.169.123:123)</Data><Data Name='TimeSourceRefId'>2074738345</Data><Data Name='CurrentStratumNumber'>4</Data></EventData><RenderingInfo Culture='en-US'><Message>The time service is now synchronizing the system time with the time source 169.254.169.123,0x9 (ntp.m|0x9|0.0.0.0:123-&gt;169.254.169.123:123) with reference id 2074738345. Current local stratum number is 4.</Message><Level>Information</Level><Task></Task><Opcode>Info</Opcode><Channel>System</Channel><Provider>Microsoft-Windows-Time-Service</Provider><Keywords></Keywords></RenderingInfo></Event>

良いですね。
生ログに近いデータが格納されているのではないでしょうか。

まとめ

関数については、データタイプを判定のうえlogEventsを抽出し、改行コードを付与した変換を行う処理となっています。

ブループリントを使うことでこういった処理が実装不要で利用可能です。
生ログを格納することで、Athena等でも使いやすくなるケースがあると思いますので、是非試してみてください。

Share this article

facebook logohatena logotwitter logo

関連記事

CDKでデプロイしたLambda Functionsで `Cannot find module '@smithy/service-error-classification'`が発生したので対応した話

CDKでデプロイしたLambda Functionsで `Cannot find module '@smithy/service-error-classification'`が発生したので対応した話

User avatar
岩田智哉
2024.12.18
API Gateway + Lambda + Pollyでテキストを音声に変換して、Flutterから再生してみた

API Gateway + Lambda + Pollyでテキストを音声に変換して、Flutterから再生してみた

User avatar
sora
2024.12.18
Lambda 関数のソースコードを S3 経由でアップロードする方法を教えてください

Lambda 関数のソースコードを S3 経由でアップロードする方法を教えてください

User avatar
hato
2024.12.17
ALB + EC2 Auto Scaling 環境下、"スケーリングプロセスの中断" と "Lambda 関数" を利用してヘルスチェックで失敗したインスタンスを保持しつつスケールアウトも実行させてみた

ALB + EC2 Auto Scaling 環境下、"スケーリングプロセスの中断" と "Lambda 関数" を利用してヘルスチェックで失敗したインスタンスを保持しつつスケールアウトも実行させてみた

User avatar
片方 裕人
2024.12.16
Classmethod's white logo
Facebook's logo
X's logo
YouTube's logo

© Classmethod, Inc. All rights reserved.